Imaging apparatus

ABSTRACT

An imaging apparatus includes a first and a second imaging unit, a clipping unit, an image processing control unit. The first imaging unit is provided on a front face of an apparatus main body. The second imaging unit is provided on a back face of the apparatus main body. The clipping unit clips image data of the second main subject from the second acquired image data. The image processing control unit combines the first acquired image data with the image data of the second main subject clipped by the clipping unit to meet combining conditions determined by the first image data.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is based upon and claims the benefit of priority fromprior Japanese Patent Application No. 2011-042871, Feb. 28, 2011, theentire contents of which are incorporated herein by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an imaging apparatus that can combineimage data acquired by so-called self-photography, i.e., photographing,e.g., a facial part of a given person who is a main subject byhimself/herself with image data acquired by an imaging unit provided ona back face of an apparatus main body.

2. Description of the Related Art

A technology of such an imaging apparatus is disclosed in, e.g., Jpn.Pat. Appln. KOKAI Publication No. 2004-147046. The gazette of Jpn. Pat.Appln. KOKAI Publication No. 2004-147046 discloses mounting a firstcamera on a first surface of a housing, mounting a second camera on asecond surface which is a back side of the first surface of the housing,photographing a user's face by using the first camera, photographing alandscape the user is watching by using the second camera, performingphotography using the first and second cameras at the same time, orcombining respective images acquired by the first camera and the secondcamera to shoot a combined picture.

BRIEF SUMMARY OF THE INVENTION

According to aspect of the invention, there is provided an imagingapparatus comprising: a first imaging unit which is provided on a frontface of an apparatus main body and images a first main subject to obtainfirst acquired image data; a second imaging unit which is provided on aback face of the apparatus main body and images a second main subject toobtain second acquired image data; a clipping unit which clips imagedata of the second main subject from the second acquired image dataobtained by imaging of the second imaging unit; and an image processingcontrol unit which combines the first acquired image data with the imagedata of the second main subject clipped by the clipping unit to meetcombining conditions determined by the first image data.

According to aspect of the invention, there is provided an imagingsystem comprising: a first imaging unit which is provided on a frontface of an apparatus main body and images a first main subject to obtainfirst acquired image data; a second imaging unit which is provided on aback face of the apparatus main body and images a second main subject toobtain second acquired image data; a clipping unit which clips imagedata of the second main subject from the second acquired image dataobtained by imaging of the second imaging unit; an image processingcontrol unit which combines the first acquired image data with the imagedata of the second main subject to meet present combining conditions,thereby creating combined image data; and a combination determining unitwhich makes at least one of a judgment upon a picture composition of thefirst acquired image data including the first main subject as thecombining conditions, a judgment whether a combination space for thesecond main subject is present in the first acquired image; and ajudgment whether combining the image data of the second main subjectclipped by the clipping unit with the first acquired image data ispossible.

According to aspect of the invention, there is provided an imagingmethod using an imaging apparatus comprising: imaging a first mainsubject by a first imaging unit provided on a front face of an imagingapparatus main body to obtain first acquired image data; imaging asecond main subject by a second imaging unit provided on a back face ofthe imaging apparatus main body to obtain second acquired image data;clipping image data of the second main subject from the second acquiredimage data; combining the first acquired image data with the clippedimage data of the second main subject to create combined image data;detecting the facial part of the first main subject from the firstacquired image data; and judging whether combining the image data of thesecond main subject clipped by the clipping unit with the first acquiredimage data is possible based on a position of the facial part in thecreation of the combined image data.

Advantages of the invention will be set forth in the description whichfollows, and in part will be obvious from the description, or may belearned by practice of the invention. The advantages of the inventionmay be realized and obtained by means of the instrumentalities andcombinations particularly pointed out hereinafter.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWING

The accompanying drawings, which are incorporated in and constitute apart of the specification, illustrate embodiments of the invention, andtogether with the general description given above and the detaileddescription of the embodiments given below, serve to explain theprinciples of the invention.

FIG. 1 is a block diagram showing a first embodiment of an imagingapparatus according to the present invention;

FIG. 2A is an outside drawing showing a front face of the apparatus;

FIG. 2B is an outside drawing showing a back face of the apparatus;

FIG. 3 is a view showing how a user performs self-photography using theapparatus;

FIG. 4 is a schematic block diagram showing a back face imaging unit inthe apparatus;

FIG. 5 is a schematic view for explaining a function for clipping imagedata of a user alone in the apparatus;

FIG. 6 is a view showing an example of an imaging situation for creatingcombined image data by the apparatus;

FIG. 7 is a schematic view showing an example of inversion of a shadedportion determined to be inappropriate for combination by the apparatus;

FIG. 8 is a schematic view showing image data of a facial part of theuser clipped from two pieces of acquired image data obtained by theapparatus when a camera main body was held with a right hand forphotography;

FIG. 9 is a schematic view showing image data which seems to beextremely unbalanced in regard to a mother and a child shown asfull-length figures and the user whose face alone is shown in theapparatus;

FIG. 10 is a schematic view for explaining a judgment upon whether thereis a space for combining image data of the user's facial part with firstacquired image data acquired by the apparatus;

FIG. 11 is a view for explaining a judgment upon whether imagecombination in the apparatus is appropriate;

FIG. 12 is a view for explaining a judgment upon whether the imagecombination in the apparatus is appropriate;

FIG. 13 is a view for explaining a judgment upon whether the imagecombination in the apparatus is appropriate;

FIG. 14 is a view showing combined image data obtained by combining theuser's image with an image of the mother and the child acquired by theapparatus;

FIG. 15 is a view showing combined image data, in which the user's righthand is excluded from a picture composition of the image to hide theright hand, acquired by the apparatus;

FIG. 16 is a view showing combined image data, from which an image dataportion of the user's right hand has been clipped, obtained by theapparatus;

FIG. 17 is a view showing an optimum inclined state at the time ofphotography in the apparatus;

FIG. 18 is a view showing an example of an inappropriate inclined stateof the camera main body at the time of photography in the apparatus;

FIG. 19 is a view showing another example of an inappropriate inclinedstate of the camera main body at the time of photography in theapparatus;

FIG. 20 is a camera control flowchart including image combination in theapparatus;

FIG. 21 is a control flowchart of slide movement in the apparatus;

FIG. 22 is a view showing an operation for the slide movement of theuser's image in the apparatus;

FIG. 23 is a view showing the slide movement of the user's image in theapparatus;

FIG. 24 is a block diagram showing functions of the apparatus in asimplified manner;

FIG. 25 is a block diagram showing a second embodiment of an imagingapparatus according to the present invention;

FIG. 26A is a front outside drawing showing the apparatus;

FIG. 26B is a back outside drawing showing the apparatus;

FIG. 27 is a view showing an imaging state when simultaneouslyperforming imaging at first timing by a main imaging unit and a backface imaging unit in the apparatus;

FIG. 28 is a view showing an imaging state when simultaneouslyperforming imaging at second timing by the main imaging unit and a backface imaging unit in the apparatus;

FIG. 29 is a view showing first acquired image data acquired by imagingusing the main imaging unit in the apparatus at timing T1;

FIG. 30 is a view showing first acquired image data acquired by imagingusing the main imaging unit in the apparatus at timing T2;

FIG. 31 is a view showing an operation of slide movement of a user'image in the apparatus; and

FIG. 32 is a view showing the slide movement of the user's image in theapparatus.

DETAILED DESCRIPTION OF THE INVENTION

A first embodiment according to the present invention will now bedescribed hereinafter in detail.

FIG. 1 is a block diagram showing an imaging apparatus (a camera). FIG.2A is an outside drawing of a front face of this apparatus. FIG. 2B isan outside drawing of a back face of this apparatus.

An image processing control unit 2 having a microcomputer mountedthereon is provided in an apparatus main body (a camera main body) 1.The image processing control unit 2 executes a previously stored cameracontrol program to effect a series of operation control, e.g., a shutteroperation, exposure processing, aperture adjustment, image processing,and others performed at the time of photography. Further, the imageprocessing control unit 2 executes control over an imaging operation forobtaining acquired image data, control over a reproducing operation forrecorded acquired image data, and others. To the image processingcontrol unit 2 are connected a main imaging unit 3 as a first imagingunit, a back face imaging unit 4 as a second imaging unit, an operationunit 5, a recording unit 6, a display unit 7, a touch panel 8, anacceleration sensor 9 as a posture sensor, and a clock unit 10.

A lens optical system 3 a of the imaging unit (which will be referred toas a main imaging unit hereinafter) is provided on a front face of thecamera main body 1 as shown in FIG. 2A. A release button 5 a as theoperation unit 5 is provided on an upper portion of the camera main body1.

The display unit 7 and a the back face imaging unit 4 are provided on aback face of the camera main body 1 as shown in FIG. 2B.

The main imaging unit 3 acquires an optical image from a subjectincluding a first main subject such as a person and outputs a firstimage signal. The main imaging unit 3 images, e.g., a family of a userwho performs photography. The main imaging unit 3 includes a lensoptical system including a focus lens, a diaphragm, a shutter, and animaging element. The main imaging unit 3 transmits an optical image froma subject to the imaging element through the lens optical system, thediaphragm, and the shutter. The imaging element converts the opticalimage from the subject into an image signal.

The back face imaging unit 4 is used for, e.g., self-photography forphotographing the user himself/herself who performs photography. Theback face imaging unit 4 acquires an optical image from the subjectincluding a second main subject like a person, e.g., the user andoutputs a second image signal. The back face imaging unit 4 includes apair of three-dimensional (3D) imaging units, i.e., a left (L) imagingunit 4 a and a right (R) imaging unit 4 b. The imaging unit 4 a and theimaging unit 4 b have a parallax. The imaging unit 4 a outputs a secondimage signal for acquiring left (L) parallax image data. The imagingunit 4 b outputs a second image signal for acquiring (R) parallax imagedata. The left (L) parallax image data and the right (R) parallax imagedata have a parallax. Therefore, the back face imaging unit 4 outputstwo second image signals for acquiring the two pieces of left and right(LR) parallax image data having the parallax.

The operation unit 5 accepts manual operations of the user. Theoperation unit 5 includes, e.g., a power supply button, a release button5 a, a reproduction button, a moving picture button, a self-photographybutton, and others. The operation unit 5 accepts operations such asswitching of a moving picture mode or a still picture mode, switching toa self-photography mode, switching to a self-photography and imagecombination mode, adjustment of a diaphragm, and others.

The recording unit 6 records acquired image data obtained by the imageprocessing control unit 2 at the time of photography, i.e., firstacquired image data obtained by performing image processing to the firstimage signal output from the main imaging unit 3 and the two pieces ofparallax image data obtained by performing image processing to the twoimage signals which are output from the back face imaging unit 4 andhave a parallax.

The acceleration sensor 9 is provided to the camera main body 1. Theacceleration sensor 9 detects a movement or a posture of the camera mainbody 1. For example, the acceleration sensor 9 which is of a triaxialtype is used to enable recognizing a level or a direction ofacceleration applied when the camera main body 1 is moved. Theacceleration sensor 9 also outputs a signal associated withgravitational acceleration. The image processing control unit 2 candetermine a posture, e.g., an inclination of the camera main body 1 byprocessing the signal associated with the gravitational accelerationoutput from the acceleration sensor 9.

In case of detecting a movement of the camera main body 1, a magneticfield sensor and the like can be also used besides the accelerationsensor 9. The magnetic field sensor determines an orientation in whichthe camera main body 1 is aimed. The image processing control unit 2uses the orientation determined by the magnetic field sensor as an aidwhen determining a movement of the camera main body 1. The movement ofthe camera main body 1 may be determined based on an output signal fromthe acceleration sensor or the magnetic field sensor.

The clock unit 10 counts passage of time.

The image processing control unit 2 executes image processing withrespect to the first image signal output from the main acquiring unit 3to acquire the first acquired image data, and executes image processingwith respect to the two second image signals output from the back faceimaging unit 4 to acquire the two pieces of parallax image data. Theimage processing control unit 2 performs image processing with respectto the two pieces of parallax image data to clip image data of thesecond main subject. The image processing control unit 2 combines thefirst acquired image data with the clipped image data of the second mainsubject to create combined image data. The image processing control unit2 has a temporary recording unit 21, a face detection unit 22, a figureclipping unit 23, a combination determining unit 24, and a combinationunit 25.

The temporary recording unit 21 temporarily records the first acquiredimage data obtained by imaging of the main imaging unit 3 and the twopieces of parallax image data obtained by imaging of the back faceimaging unit 4.

The face detection unit 22 detects a facial part of, e.g., a person asthe first main subject from the first acquired image data. The facedetection unit 22 detects arrangement of each part in a facial part of aperson and the like e.g., an ocular part, a nasal part, or an oral part.

The figure clipping unit 23 clips image data of the second main subjectfrom the two pieces of parallax image data.

Description will now be given as to clipping of image data of a facialpart of the user himself/herself performed by the figure clipping unit23.

FIG. 3 shows how a user H photographs himself/herself. The user H holdsthe camera main body 1 with, e.g., his/her right hand RH to photographhimself/herself. The camera main body 1 shows respective view angles 4a-g and 4 b-g of the left and right (LR) imaging units 4 a and 4 bcorresponding to the back face imaging unit 4.

FIG. 4 is a schematic structural view of the back face imaging unit 4.The back face imaging unit 4 is a 3D camera as described above andincludes two taking lenses 30 and 31 and two imaging elements 32 and 33.The taking lens 30 is associated with the imaging element 32, and thetaking lens 31 is associated with the imaging element 33. Each of thetaking lenses 30 and 31 is arranged to be apart from each other by aninterval X1.

Each of two pieces of acquired image data K1 and K2 shown in FIG. 5 isparallax image data. Each parallax image data represents an image of theuser H who performs, e.g., photography from respective positions of thetaking lenses 30 and 31 apart from each other by the interval X1. Therespective pieces of acquired image data K1 and K2 have a parallaxcorresponding to the interval X1 between the respective taking lenses 30and 31. When the user H and a background B have a difference indistance, a relative position that respective images of the user H andthe background B strike on each imaging element 32 or 33 through eachtaking lens 30 or 31 based on the triangulation principle differs. Thatis, a difference between respective incidence positions of the user Hand the background B is X11 on one imaging element 32. A differencebetween respective incidence positions of the user H and the backgroundB is X12 on the other imaging element 33. Each of the differences X11and X12 is a parallax associated with the interval X1 between the takinglenses 30 and 31. The respective differences X11 and X12 are differentfrom each other and meet a relationship of X11≠X12,

Therefore, when the 3D back face imaging unit 4 is used to performphotography with the parallax, for example, such two pieces of acquiredimage data K1 and K2 as shown in FIG. 5 are obtained. As shown in thedrawing, when the two pieces of acquired image data K1 and K2 arearranged while fitting positions of respective image data portions of afacial part of the user H to each other, the respective pieces of imagedata of the user H have a high degree of coincidence in the two piecesof acquired image data K1 and K2, but the respective pieces of imagedata of a background part have a low degree of coincidence. Therefore,when the image data having the high degree of coincidence is clippedfrom any one of the two pieces of acquired image data K1 and K2, imagedata L of the facial part of the user H alone can be clipped.

The image combination unit 25 combines the image data L of the facialpart of the user H clipped by the figure clipping unit 23 with the firstacquired image data obtained by imaging of the main imaging unit 3 underpreset combining conditions.

The combination determining unit 24 judges whether the image data L ofthe facial part of the user H clipped by the figure clipping unit 23 canbe combined with the first acquired image data based on, e.g., a picturecomposition of the first acquired image data including a family of theuser H as the combining conditions for obtaining the first acquiredimage data.

The combination determining unit 24 makes as the combining conditions atleast one of a judgment upon a position of a shaded portion, e.g., thefacial part of the user H, a judgment upon presence/absence of acombination space for the facial part and the like of the user H basedon a position of the family of the user H in the first acquired imagedata, a judgment upon acquisition of the facial part of the user H byimaging, and a judgment upon whether, e.g., a facial part of the familyof the user H and the facial part of the user H have been imaged at thesame height position.

The combination determining unit 24 makes as the combining conditions atleast one of a judgment upon a picture composition of the first acquiredimage data including the first main subject such as a person, a judgmentupon presence/absence of a combination space for the second main subjectsuch as a person like the user in the first acquired image, and ajudgment upon whether image data of the second main subject clipped bythe figure clipping unit 23 can be combined with the first acquiredimage data.

The combination determining unit 24 makes a judgment uponpresence/absence of the combination space for a space used forcombination of the image data of the second main subject clipped by thefigure clipping unit 23 based on a position of the first main subjectdetected by the face detection unit 22, and makes a judgment uponwhether combination of the image data of the second main subject ispossible based on a result of this judgment.

A judgment when the combination is inappropriate will now be described.

First, a first combination judgment will be described.

FIG. 6 shows an example of an imaging situation for creating combinedimage data. Here, the user H is a father F and a mother M and a child Cform his family to make a usage scene understandable. In the followingdescription, the user H will be referred to as a user F since he is thefather F.

The user F holds the camera main body 1 and aims the main imaging unit 3at the mother M and the child C to perform an imaging operation, therebyobtaining first acquired image data. At the same time, the user F aimsthe respective left and right (LR) imaging units 4 a and 4 b as the backface imaging unit 4 at the user F himself and performs the imagingoperation to obtain two pieces of parallax image data, e.g., two piecesof acquired image data K1 and K2 shown in FIG. 5. As a photographysituation at this moment, sunlight with high illuminance is applied fromthe sun. Respective shaded portions 30 h, 30 m, and 30 c are formed atobliquely lower right portions of the facial parts of the user F, themother M, and the child C in the drawing because of the application ofsunlight.

Combined image data G1 is obtained by combining image data of the facialpart of the user F clipped from each of the two pieces of acquired imagedata K1 and K2 with the first acquired image data including the mother Mand the child C acquired by photography in the above-describedphotography situation.

FIG. 7 is a schematic view of the combined image data G1. In thecombined image data G1, positions of the respective shaded portions 30 mand 30 c of the mother M and the child C are opposite to a position ofthe shaded portion 30 h of the user F. The respective shaded portions 30m and 30 c of the mother M and the child C are formed at the obliquelylower right portions of the facial parts. However, the shaded portion 30h of the user F is inverted to be unnaturally formed at an obliquelylower left portion of the facial part.

Therefore, the combination determining unit 24 detects the positions ofthe respective shaded portions 30 m and 30 c of the mother M and thechild C and the position of the shaded portion 30 h of the user F in thecombined image data G1 shown in FIG. 7, for example. The combinationdetermining unit 24 compares the positions of the respective shadedportions 30 m, 30 c, and 30 h with positions of the respective facialparts of the mother M, the child C, and the user F and judges whetherthe positions of the respective shaded portions 30 m, 30 c, and 30 h arenot inverted. When the inversion is found as a result of the judgment,the combination determining unit 24 determines that the combination isinappropriate.

The combination determining unit 24 detects each portion whose luminanceis reduced to a predetermined luminance level or a lower level ascompared with the surroundings as the shaded portion 30 m, 30 c, or 30h. The face detection unit 22 detects the positions of the respectivefacial parts of the mother M, the child C, and the user F. Aninclination of shade can be determined by using, e.g., the facedetection unit 22 to compare left and right brightness distributions ofthe face with each other.

A second combination judgment will now be described.

For example, as shown in FIG. 6, the user F holds the camera main body 1with his right hand to perform a photography operation. Image data ofthe facial part of the user F clipped from the two pieces of theacquired image data K1 and K2 by this photography operation represents,e.g., an image of an upper body in which a right arm Fr is projected asshown in FIG. 8.

On the other hand, when the first acquired image data is obtained byphotographing full-length figures of the mother NM and the child C, suchcombined image data G2 as shown in FIG. 9 can be obtained by combiningthe image data of the facial part of the user F clipped from the twopieces of the acquired image data K1 and K2 with the first acquiredimage data. In the combined image data G2, the mother M and the child Cas the full-length figures are combined with the user F showing thefacial part alone. The combined image data G2 may possibly represent animage that the main subject is extremely unbalanced.

Therefore, for example, the combination determining unit 24 detects therespective positions of the facial parts of the mother M and the child Cand the position of the facial part of the user F in the combined imagedata G2 shown in FIG. 9 and judges whether the respective positions ofthese facial parts are present at substantially the same height in thehorizontal direction. When the positions of the respective facial partsare not present at substantially the same height, the combinationdetermining unit 24 determines that the combination is inappropriate.The horizontal direction can be obtained from a result of detection of aposture of the camera main body 1 performed by the acceleration sensor9.

A third combination judgment will now be described.

FIG. 10 is a schematic view of an example of first acquired image dataS1. In the first acquired image data S1, the most part of image data isoccupied by the mother M and the child C. The first acquired image datadoes not have a space for combination of image data of the facial partof the user F.

Therefore, for example, the combination determining unit 24 calculatesan occupancy ratio of image data of the mother M and the child C in thefirst acquired image data S1 shown in FIG. 10 to compute an area of thespace for combination of the image data of the facial part of the userF. The combination determining unit 24 compares the area of the spacewith an area occupied by the image data of the facial part of the user Fand judges whether the space for combination of the image data of thefacial part of the user F is present in the first acquired image dataS1. If the space is not present as a result of this judgment, thecombination determining unit 24 determines that the combination isinappropriate.

A judgment when the image combination is appropriate will now bespecifically explained with reference to FIG. 11 to FIG. 13.

As shown in FIG. 11, for example, as first acquired image data S2 of themother M and the child C acquired by imaging of the main imaging unit 3,it is preferable to photograph respective upper bodies of the mother Mand the child C to obtain this image data S2 like the image data of thefacial part of the user F obtained by imaging of the back face imagingunit 4. The upper body of each of the mother M and the child C isapproximately triple a height of each facial part.

Therefore, as shown in FIG. 12, a distance Yf, which is a distance froma central portion of the facial part of the motor M to a lower end ofthe image data, is allowed to the extent of the following level:

Yf≦2.5Df  (1)

As shown in FIG. 12, combined image data G3 requires a width space forcombination of image data corresponding to a shoulder width Hk of theuser F. If the shoulder width Hk of the user F is approximately double aheight Df of the facial part of the mother M, a width space representedby the following expression or a larger space is required on, e.g., theleft side of the mother M:

Hf≈2×Df  (2)

To meet this necessary condition, the following condition is required:

Xf>3×Df  (3)

Xf indicates a distance between, e.g., a left end of the image data anda central portion of the facial part of the mother M.

Therefore, the face detection unit 22 detects a position of the centralportion of the facial part of the mother M.

The combination determining unit 24 receives information indicative ofthe position of the central portion of the facial part of the motor Mdetected by the face detection unit 22, obtains the distance Xf betweenthe central portion of the facial part of the mother M and the left endof the first acquired image data S2, and also obtains the height (alength) Df of the facial part of the mother M.

The combination determining unit 24 judges whether the condition ofExpression (3) is met. When the condition of Expression (3) is met, thecombination determining unit 24 determines that the space forcombination of the image data of the facial part of the user F ispresent in the first acquired image data S1.

FIG. 13 shows combined image data G4 obtained by combining the image ofthe user F with an image of the child C. Images of the mother M and thechild C in the combined image data G4 are shown smaller than, e.g., theimages of the mother M and the child C depicted in FIG. 11.

To create the combined image data G, the combination determining unit 24judges whether the space for combination of the image data of the facialpart of the user F is present in, e.g., the first acquired image databefore the combination of the image data facial part of the user F. Whenthe space is determined to be present, the combination determining unit24 reduces a size of the image data of the facial part of the user F andcombines the reduced image data with the first acquired image data inaccordance with sizes of the images of the mother M and the child C.

FIG. 14 shows combined image data G5 obtained by combining the images ofthe mother M and child C with the image of the user F. The user F in thecombined image data G5 is shown in such a manner that his right hand Frholding the camera main body 1 is unnatural. In such a case, the imagecombination unit 25 executes image processing to remove the right handFr of the user F from the picture composition like combined image dataG6 shown in FIG. 15 and hide the right hand Fr. As a result,unnaturalness of the right hand Fr of the user F can be eliminated.

In this case, a height Df1 of the facial part of the user F is set to beequal to the height Df of the facial part of the mother M (Df1=Df).Further, a distance Xf1 between, e.g., the left end of the image dataand a central portion of the facial part of the user F is set to beequal to the height Df1 of the facial part of the user F (Xf1=Df1). Adistance Yf between, e.g., a lower end of the image data and the centralportion of the facial part of the user F is set to be equal to adistance Yf1 between, e.g., the lower end of the image data and thecentral portion of the facial part of the mother M (Yf=Yf1). As aresult, unnaturalness can be eliminated.

Furthermore, these relationships Df1=Df, Xf1=Df1, and Yf1=Yf areslightly increased on the assumption that the user F performsphotography. The unnaturalness of the right hand Fr of the user F can beassuredly eliminated by slightly increasing these relationships Df1=Df,Xf1=Df1, and Yf1=Yf, e.g., multiplying these relationships by 1.1,namely, attaining the following expressions:

Df1=1.1×Df

Xf1=1.1×Df1

Yf1=1.1×Yf

In this case, it is assumed that images of the mother M and the child Dare obtained by shooting shoulder parts to the facial parts of themother M and the child C.

Therefore, the combination determining unit 24 executes image processingwith respect to the combined image data G5 depicted in FIG. 14 toachieve the following relationship:

Df1=1.1×Df

Xf1=1.1×Df1

Yf1=1.1×Yf

In the combined image data G6 depicted in FIG. 15, if the distance Yfbetween, e.g., the lower end of the image data and the central portionof the facial part of the user F is equal to the height (the length) Df1of the facial part of the user F (Yf=Df1), the right hand Fr of the userF is removed from the picture composition of the image, and theunnaturalness of the right hand Fr of the user F can be eliminated.

Even in case of the combined image data G5 in which the right hand Fr ofthe user F is unnatural as shown in FIG. 14, it is possible to obtaincombined image data in which the unnaturalness of the right hand Fr ofthe user F is eliminated by clipping an image data portion G5 arepresenting a picture configuration where the right hand Fr of the userF is shown as an image from the combined image data G5 as shown in FIG.16.

An inclination of the camera main body 1 for creating combined imagedata will now be described with reference to FIG. 17 to FIG. 19.

FIG. 17 shows an optimum inclined state of the camera main body 1 at thetime of photography. It is appropriate to arrange a photography opticalaxis Q of the camera main body 1 on a straight line with respect to theuser F, the mother M, and the child C so that the user F can face themother M, and the child C through the camera main body 1. As the bestphotography situation, the camera main body 1 is not inclined, the userF, the mother M, and the child C are arranged at substantially the sameheight positions, and the photography optical axis Q of the camera mainbody 1 is horizontally arranged. Whether the photography optical axis Qof the camera main body 1 is horizontally arranged can be judged byusing the image processing control unit 2 to obtain an inclination ofthe camera main body 1 from an orientation signal output from theacceleration sensor 9.

On the other hand, FIG. 18 shows an example of an inappropriate inclinedstate of the camera main body 1 at the time of photography. Thephotography optical axis Q of the camera main body 1 is horizontallyarranged. On the other hand, the user F and the child C have differentheights. For example, when a height position of the camera main body 1is conformed to the child C, the camera main body 1 can image the facialpart of the child C by the main imaging unit 3. However, the back faceimaging unit 4 images a chest part of the user F. Therefore, the facialpart of the user F cannot be imaged, which is unnatural.

Therefore, the combination determining unit 24 detects a differencebetween acquired image data of the facial part of the child C obtainedby imaging of the main imaging unit 3 and image data of the chest partof the user F obtained by imaging of the back face imaging unit todetermine that combination is inappropriate.

FIG. 19 shows another example of an inappropriate inclined state of thecamera main body 1 at the time of photography. The photography opticalaxis Q of the camera main body 1 is arranged to be inclined with respectto the horizon. An inclination θ of the camera main body 1 is formed toconnect the respective facial parts of the user F and the child C havinga difference in height with each other on a straight line. In this case,for example, the ground is imaged in image data of the facial part ofthe child C obtained by imaging of the main imaging unit 3. Therefore,when combined image data is created, the facial part of the user F maybe arranged on, e.g., the ground, which is unnatural.

Therefore, the combination determining unit 24 obtains the inclination θof the camera main body 1 from an orientation signal output from theacceleration sensor 9 and determines that the combination isinappropriate if the camera main body 1 has the inclination θ.

If the image data of the facial part of the user F clipped by the figureclipping unit 23 cannot be combined with the first acquired image dataas a result of the judgment of the combination determining unit 24,information indicating that the combination is impossible, e.g.,information indicating that the combination of the image data of thefacial part of the user F is impossible is displayed in, e.g., thedisplay unit 7.

The image processing control unit 2 judges whether the combination ofthe image data of the facial part of the user F clipped by the figureclipping unit 23 is possible based on positions of the mother M and thechild C detected by the face detection unit 22.

The image processing control unit 2 obtains a posture of the camera mainbody 1, e.g., the inclination θ of the camera main body 1 from anorientation signal output from the acceleration sensor 9. The imageprocessing control unit 2 judges whether the mother M, the child C, andthe user F are imaged at the same height position based on theinclination θ of the camera main body 1.

When the combination is determined to be possible as a result of thejudgment made by the combination determining unit 24, the combinationunit 25 combines the imaged data of the facial part of the user Fclipped by the figure clipping unit 23 with the first acquired imagedata recorded in the temporary recording unit 21.

The image processing control unit 2 expands or contracts the image dataof the facial part of the user F clipped by the figure clipping unit 23to be combined with the first acquired image data.

The image processing control unit 2 can slidably move the image data ofthe facial part of the user F clipped by the figure clipping unit 23into the first acquired image data.

Camera control including image combination using the camera will now bedescribed with reference to a camera control flowchart depicted in FIG.20.

An operation mode of the camera main body 1 is set by an operation ofthe operation unit 5 effected by the user. At a step S101, the imageprocessing control unit 2 reads various kinds of buttons on theoperation unit 5 for user operations and judges whether the operationmode is a photography mode. If the operation mode is the photographymode as a result of this judgment, the image processing control unit 2judges whether an image of the facial part of the user F is present in,e.g., two pieces of parallax image data obtained by imaging of the backface imaging unit (the respective L and R imaging units 4 a and 4 b)provided on the back face of the camera main body 1.

If the image of the facial part of the user F is not present in, e.g.,the two pieces of parallax image data as a result of this judgment, thecombination determining unit 24 determines that image data of the facialpart of the user F clipped by the figure clipping unit 23 can becombined with the first acquired image data obtained by imaging of themain imaging unit 3.

At a step S103, the image processing control unit 2 executes imageprocessing with respect to an image signal output from the main imagingunit 3 and displays a processed signal as a through image in the displayunit 7.

At a step S104, the image processing control unit 2 judges whether aphotography start instruction has been issued in response to anoperation of the operation unit 5 by the user. When the photograph startinstruction has been issued as a result of this judgment, the imageprocessing control unit 2 executes still picture shooting for obtainingstill image data by operating the imaging unit 2 in case of, e.g.,shooting of a still image at a step S105. In case of shooting of amoving picture, the image processing control unit 2 executes movingpicture shooting for obtaining moving image data by continuouslyoperating the main imaging unit 3.

After end of the still picture shooting or the moving picture shooting,the image processing control unit 2 creates a still picture or movingpicture image file and records the created image file in the recordingunit 6 at a step S106. After the creation of the image file, the imageprocessing control unit 2 displays the still picture or moving picturedata in the display unit 7.

At a step S108, the image processing control unit 2 then judges whethera power supply OFF instruction has been issued in response to anoperation of the operation unit 5 effected by the user. If the powersupply OFF instruction has been issued, the image processing controlunit 2 terminates the camera control. If the power supply OFFinstruction has not been issued, the image processing control unit 2returns to the step S101.

If the operation mode of the camera 1 is not the photography mode, theimage processing control unit 2 advances to a step S111 from the stepS101 to judge whether the operation mode is a reproduction mode. If theoperation mode is the reproduction mode as a result of this judgment,the image processing control unit 2 lists (thumbnails) in the displayunit 7 respective files of still pictures and moving pictures recordedin the recording unit 6. When any one of the respective files of stillpictures and moving pictures is touched and selected on the display unit7 by the user, the touch panel 8 outputs a touch position signalassociated with a touch position at a step S113.

At a step S114, the image processing control unit 2 reproduces in thedisplay unit 7 the still picture or the moving picture selected by touchfrom the respective files of still pictures and moving pictures listedin the display unit 7 from the touch position signal from the touchpanel 8.

If no file of still pictures and moving pictures is selected by touch ofthe user, the image processing control unit 2 terminates thereproduction mode at a step S115 and judges whether the power supply OFFinstruction has been issued at the next step S108. If the power supplyOFF instruction has not been issued, the image processing control unit 2again accepts a change operation for the photography mode or thereproduction mode and returns to the step S101.

On the other hand, at the step S102, if the image of the facial part ofthe user F is present in the two pieces of parallax image data obtainedby imaging of the back face imaging unit (the respective LR imagingunits 4 a and 4 b) provided on the back face of the camera main body 1,the image processing control unit 2 advances to a step S121. At the stepS121, the image processing control unit 2 obtains a posture of thecamera main body 1, e.g., the inclination θ of the camera main body 1from an orientation signal output from the acceleration sensor 9 andalso judges whether the inclination of the camera main body 1 ishorizontal.

As a result of this judgment, if the image of the facial part of theuser F is present in the two pieces of parallax image data, theinclination θ of the camera main body 1 is horizontal, the camera mainbody 1 is in an optimum inclined state at the time of shooting as shownin, e.g., FIG. 17, namely, the camera main body 1 is not inclined, theuser F, the mother M, and the child C are present at substantially thesame heights, and the photography optical axis Q of the camera main body1 is horizontally arranged, the image processing control unit 2 judgeswhether a person is present in front of the camera main body 1, i.e., inthe first acquired image data, e.g., first acquired image data S2 shownin FIG. 11 obtained by imaging of the main imaging unit 3. Here, themother M and the child C are determined to be present.

If the inclination of the camera main body 1 is not horizontal, theimage processing control unit 2 advances to a step S127, and displays awarning that the inclination of the camera main body 1 is not horizontalin, e.g., the display unit 7, or gives this warning using sound.

When the mother M and the child C are present in this manner, the imageprocessing control unit 2 determines a height position of the highestfacial part in the first acquired image data S2. Here, the facedetection unit 22 detects the respective facial parts of the mother Mand the child C from the first acquired image data S1. The imageprocessing control unit 2 obtains heights and positions of therespective facial parts of the mother M and the child C detected by theface detection unit 22 and determines the facial part present at thehighest position in the first acquired image data, which is the heightand the position of the mother M in this example, from the obtainedheights and positions of these facial parts.

At a step S124, assuming that Yf is a distance from the central portionof the facial part of the mother M to the lower end of the image dataand Df is the height (a length) of the facial part of the mother M, theimage processing control unit 2 judges whether the distance Yf meetsYf≦2.5Df in Expression (1).

If Expression (1) is met as a result of this judgment, the imageprocessing control unit 2 determines a person who is present at the leftend in the first acquired image data S2 shown in FIG. 11, which is theposition of the facial part of the mother M in this example, at a stepS125. The position of the facial part of the mother M has been alreadydetected by the face detection unit 22.

At a step S126, the image processing control unit 2 judges whetherXf>3×Df in Expression (3) is met since a width space for combination ofimage data of the shoulder width Hk of the user F is required in thecombined image data G3 as shown in FIG. 12.

As a result of this judgment, if Expression (3) is met, the imageprocessing control unit 2 instructs the figure clipping unit 23 to clipthe image data of the facial part of the user F at a step S131. Thefigure clipping unit 23 clips the image data of the facial part of theuser F from the two pieces of parallax image data obtained by imaging ofthe back face imaging unit (the respective LR imaging units 4 a and 4b). The image combination unit 25 combines the image data of the facialpart of the user H clipped by the figure clipping unit 23 with the firstacquired image data obtained by imaging of the main imaging unit 3.

The image processing control unit 2 conforms the height (the length) Df1of the facial part of the user F from the clipped image data of thefacial part of the user F to 1.1-fold of the height (the length) Df ofthe facial part of the mother M, i.e., the following expression:

Df1=1.1×Df

At a step S132, the image processing control unit 2 sets the height (thelength) Df1 of the facial part of the user F to be equal to the height(the length) Df of the facial part of the mother M (Df1=Df) likecombined image data G6 shown in FIG. 15, for example. Further, the imageprocessing control unit 2 sets the distance Xf1 between, e.g., the leftend and the central portion of the facial part of the user. F in theimage data to be equal to the height (the length) Df1 of the facial partof the user F (Xf1=Df1). The image processing control unit 2 sets thedistance Yf between, e.g., the lower end and the central portion of thefacial part of the user F in the image data to be equal to the distanceYf1 between, e.g., the lower end and the central portion of the facialpart of the mother M (Yf=Yf1). As a result, as shown in FIG. 12 or FIG.15, the combined image data G3 in which the unnaturalness of the righthand Fr of the user F is eliminated can be obtained by performing theimage processing to remove the image of the right hand Fr of the user Fand hide the right hand Fr.

At a step S133, the image processing control unit 2 executes the imageprocessing with respect to the combined image data G3 depicted in FIG.12 or FIG. 13 and displays an obtained result as a through image in thedisplay unit 7.

At a step S134, the image processing control unit 2 receives a touchposition signal obtained when the user F performs a touch operation onthe touch panel 8 and judges whether the image of the user F is to bemoved in a sliding manner in combined image data G7 shown in, e.g., FIG.22 based on this touch position signal.

FIG. 21 shows a control flowchart of the slide movement. As shown inFIG. 22, the user F touches the image of the user F which is to be movedin the sliding manner on the screen of the display unit 7. The touchpanel 8 outputs the touch position signal associated with this touchposition in response to the touch of the user.

At a step S201, the image processing control unit 2 obtains the touchedposition in the image on the display unit 7 from the touch positionsignal from the touch panel 8 and judges whether this touch position isassociated with the image of the facial part of the user F.

If the touch position is associated with the image of the facial part ofthe user F as a result of this judgment, the image processing controlunit 2 traces the slide movement of the touch position from the touchposition signal output from the touch panel 8 at a step S202. That is,the user F moves the touch position to a desired position as depictedin, e.g., FIG. 23. The image processing control unit 2 moves the imageof the user F on the image of the combined image data G3 in accordancewith the slide movement of the touch position.

After the slide movement of the image of the user F, the imageprocessing control unit 2 judges whether the condition Xf>3×Dfrepresented by Expression (3) is met to determine whether a width spacefor combination of the image data of the user F is present at a stepS202.

If the condition Xf>3×Df is met as a result of this judgment, the imageprocessing control unit 2 combines the image data of the user F with theimage of the combined image data G3 at a position after the slidemovement and then advances to the step S104.

If the condition Xf>3×Df is not met, the image processing control unit 2advances to a step S203 and judges whether the distance Yf between,e.g., the lower end and the central portion of the facial part of theuser F in the image data is smaller than the height (the length) Df1 ofthe facial part of the user F (Yf≦Df1) in the combined image data G6 asshown in, e.g., FIG. 15. If Yf≦Df1 is met as a result of this judgment,the image processing control unit 2 combines the image data of the userF with the image of the combined image data G3 at the position after theslide movement at the step S202. If Yf≦Df1 is not met, the imageprocessing control unit 2 displays a warning indicating that thecombination is impossible in the display unit 7 or gives the warningusing sound at a step S205, for example.

FIG. 24 is a block diagram showing functions of the camera main body 1in a simplified manner. The camera main body 1 has the image processingcontrol unit 2, the main imaging unit 3, the back face imaging unit 4,the operation unit 5, the display unit 7, the figure clipping unit 23,and the combination unit 25. The image processing control unit 2 decidesa combining position of the image data of the facial part of the user Fin the first acquired image data.

The back face imaging unit 4 may not include the two imaging elements 32and 33 and the like for the 3D camera, and it may include one imagingelement. In case of the camera main body 1 including one imagingelement, it is sufficient to acquire two pieces of parallax image datahaving a parallax at different timings as will be described later.

As explained above, according to the first embodiment, the main imagingunit 3 is used to obtain the first acquired image data formed byshooting, e.g., the family of the user F, the back face imaging unit 4is used to obtain the two pieces of parallax image data having theparallax formed by shooting, e.g., the user F, the image data of thefacial part of the user F is clipped from the two pieces of parallaximage data, and the clipped image data of the facial part of the user Fis combined with the first acquired image data obtained by shooting thefamily of the user F under the preset combining conditions.

As the combining conditions, there are, e.g., a picture composition ofthe first acquired image data including the family of the user F, aposition of a shaded portion on the facial part and the like of the userF, presence/absence of the combination space for the facial part and thelike of the user F based on, e.g., a position of the family of the userF in the first acquired image data, whether the facial part of the userF has been imaged and acquired, whether a facial part of the family ofthe user F and the facial part of the user F have been imaged at thesame height position, and others.

As a result, when viewing a combined picture of an image, for example,it is possible to obtain the combined picture giving an impression thatthe user F is fitted in the family (the mother M or the child C) or alandscape without unnaturally combining the user F with his family (themother M or the child C) or the landscape and to leave an impression ofa natural picture composition in terms of arrangement positions of theuser and the family and others.

Specifically, it is possible to determine that the combination isinappropriate.

Firstly, the combination determining unit 24 compares positions of therespective shaded portions 30 m and 30 c of the mother m and the child cwith a position of the shaded portion 30 h of the user F in the combinedimage data G1 shown in FIG. 7, for example. If the positions of therespective shaded portions 30 m, 30 c, and 30 h are inverted, thecombination determining unit 24 determines that the combination isinappropriate.

Secondly, the combination determining unit 24 judges whether respectivepositions of the facial parts of the mother M and the child C and aposition of the facial part of the user F are present at substantiallythe same heights in the horizontal direction in the combined image dataG2 depicted in FIG. 9, for example. If these positions are not presentat substantially the same heights as a result of this judgment, thecombination determining unit 24 determines that the combination isinappropriate.

Thirdly, the combination determining unit 24 compares, e.g., a region ofthe space for combination of the image data of the facial part of theuser F with a region of the facial part of the user F in the firstacquired image data S1 depicted in FIG. 10 and judges whether the spacefor combination of the image data of the facial part of the user F ispresent in the first acquired image data S1, for example. If the spaceis not present as a result of this judgment, the combination determiningunit 24 determines that the combination is inappropriate.

Subsequently, a judgment when the combination is appropriate can bemade.

The combination determining unit 24 obtains the distance Xf between thecentral portion of the facial part of the mother M and the left end ofthe first acquired image data, obtains the height (the length) Df of thefacial part of the mother M, and determines that the space forcombination of the image data of the facial part of the user F ispresent in the first acquired image data S1 if the condition (Xf>3×Df)of Expression (3) is met, thereby enabling the combination of the imagedata of the facial part of the user F.

In regard to, e.g., the unnatural right hand Fr of the user F holdingthe camera main body 1, the unnaturalness of the right hand Fr of theuser F can be eliminated by executing the image processing to hide theright hand Fr. In this case, the unnaturalness of the right hand Fr ofthe user F can be assuredly eliminated by setting relationships betweenthe height Df1 of the facial part of the user F, the height (the length)Df of the facial part of the mother M, the distance Xf1 between, e.g.,the left end and the central portion of the facial part of the user F inthe image data, the height (the length) Df1 of the facial part of theuser F, the distance Yf between, e.g., the lower end and the centralportion of the facial part of the user F in the image data, and thedistance Yf1 between, e.g., the lower end and the central portion of thefacial part of the mother M in the image data to Df1=1.1×Df,Xf1=1.1×Df1, and Yf1=1.1Yf.

Subsequently, an appropriate inclined state of the camera main body 1can be judged. That is, the best photography situation is that thephotography optical axis Q of the camera main body 1 is arranged on thestraight line with respect to the user F, the mother M, and the child Cand the user F is arranged to face the mother M and the child C throughthe camera main body 1.

Therefore, the image processing control unit 2 obtains the inclination θof the camera main body 1 from the orientation signal output from theacceleration sensor 9, and it can determine that the combination isinappropriate when the camera main body 1 has the inclination θ.

A second embodiment according to the present invention will now bedescribed with reference to the drawings.

FIG. 25 is a block diagram of an imaging apparatus. FIG. 26A is a frontoutside drawing of the apparatus. FIG. 26B is a back outside drawing ofthis apparatus. It is to be noted that like reference numerals denoteparts equal to those in FIG. 1, FIG. 2A, and FIG. 2B to omit detaileddescription thereof.

A back face imaging unit 30 is a two-dimensional camera. The back faceimaging unit 30 acquires an optical image from a subject including asecond main subject such as a person and outputs a second image signallike a main imaging unit 3. The back face imaging unit 30 images, e.g.,a user F himself/herself who performs photography. The back face imagingunit 30 comprises a lens optical system including a focus lens, adiaphragm, a shutter, and an imaging element. The imaging elementconverts the optical image transmitted from the subjected through thelens optical system, the diaphragm, and the shutter into an imagesignal.

A display unit 31 includes, e.g., a liquid crystal display. The displayunit 31 displays acquired image data obtained at the time of photographyin the liquid crystal display. The display unit 31 reproduces anddisplays the recorded acquired image data in the liquid crystal display.A touch panel 8 is provided on an upper surface of the liquid crystaldisplay. The display unit 31 can display a three-dimensional (3D) imageby displaying respective pieces of left and right (LR) image data havinga parallax in the liquid crystal display at the same time.

A guidance operation button 32 is a user operation button used fordisplaying on the liquid crystal display of, e.g., a display 6 aguidance showing a written operation method when combining firstacquired image data obtained by imaging of the main imaging unit 3 withsecond acquired image data obtained by imaging of the back face imagingunit 30. The guidance of the operation method includes, e.g., “pleasemove the camera main body sideways and do shooting again”. The guidance“please move the camera main body sideways and do shooting again” of theoperation method is displayed in the liquid crystal display of thedisplay 6 when performing imaging by the back face imaging unit 30. Theguidance “please move the camera main body sideways and do shootingagain” may be given by sound from a sound device such as a speaker.

Camera control including image combination using the camera will now bedescribed.

An image processing control unit 2 reads various kinds of buttons foruser operations with respect to an operation unit 5 and judges whetheran image combination mode is set. When the image combination mode isset, the image processing control unit 2 simultaneously executes imagingwith the main imaging unit 3 and the back face imaging unit 30. When arelease button is pressed at, e.g., timing T1 as shown in FIG. 27, themain imaging unit 3 and the back face imaging unit 30 simultaneouslyexecute imaging at the time of this pressing operation. The main imagingunit 3 images, e.g., a mother M and a child C whose are family membersof a user F. At the same time, the back face imaging unit 30 images,e.g., a facial part of the user F.

When the simultaneous imaging of the main imaging unit 3 and the backface imaging unit 30 is terminated and a guidance operation button 32 ispressed, the image processing control unit 2 displays, e.g., theguidance “please move the camera main body sideways and do shootingagain” in the liquid crystal display of the display 6. The user Fhorizontally moves a camera main body 1 in, e.g., a direction of anarrow N as shown in FIG. 28. When the horizontal movement is terminatedand the release button is pressed at timing T2 after the timing T1, themain imaging unit 3 and the back face imaging unit 30 simultaneouslycarry out imaging at the time of this pressing operation. The mainimaging unit 3 images, e.g., the mother M and the child C who are thefamily members of the user F. At the same time, the back face imagingunit 30 images, e.g., the facial part of the user F.

Respective pieces of acquired image data obtained by imaging of the mainimaging unit 3 at the respective timings T1 and T2 have a parallax. Thisparallax is associated with an interval between respective imagingpositions of the main imaging unit 3 at the respective timings T1 andT2. FIG. 29 shows first acquired image data J1 obtained by imaging atthe timing T1. FIG. 30 shows first acquired image data J2 obtained byimaging at the timing T2. In the respective pieces of acquired imagedata J1 and J2, positions of the mother M and the child C are differentin images.

Respective pieces of second acquired image data obtained by imaging bythe back face imaging unit 30 at the respective timings T1 and T2likewise have a parallax associated with an interval between therespective imaging positions of the back face imaging unit 30 at thetimings T1 and T2.

The image processing control unit 2 clips pieces of image data of themother M and the child C from the respective pieces of first acquiredimage data J1 and J2 obtained by imaging of the main imaging unit 3 atthe timings T1 and T2. Further, the image processing control unit 2clips pieces of image data of the facial part of the user F from therespective pieces of first acquired image data obtained by imaging bythe back face imaging unit 30 at the respective timings T1 and T2.

Since the image data of the mother M and the child C and the image dataof the facial part of the user F are clipped, these pieces of image datacan be separated from, e.g., the first acquired image data J1 and aremovable on the first acquired image data J1.

Therefore, the image processing control unit 2 moves the image data ofthe user F on the first acquired image data J1 in a sliding manner inaccordance with a control flowchart of slide movement depicted in FIG.21. Specifically, the user F touches the image of the user F that is tobe moved in the sliding manner in a liquid crystal display screen of adisplay unit 7 as shown in FIG. 31. The user F moves a touch position toa desired position in the sliding manner as shown in, e.g., FIG. 32.

At the time of this slide movement, the image processing control unit 2judges whether a condition Xf>3×Df represented by Expression (3) is metto determine whether a width space for combination of the image data ofthe user F is present. If the condition Xf>3×Df is met as a result ofthis judgment, the image processing control unit 2 combines the imagedata of the user F with the image of the combined image data G3 at aposition after the slide movement.

As described above, according to the second embodiment, the back faceimaging unit 30 is used as a two-dimensional camera, the camera mainbody 1 is moved sideways, and imaging is carried out at the respectivedifferent timings T1 and T2. The main imaging unit 3 and the back faceimaging unit 30 can obtain the two pieces of first and second acquiredimage data having the parallax, and the image data of the facial part ofthe user F and the image data of the mother M and the child C can beclipped from the pieces of first and second acquired image data. As aresult, the clipped image data of the facial part of the user F can becombined with the first acquired image data obtained by imaging thefamily members of the user F under the preset combining conditions.Consequently, according to the second embodiment, the same effect asthat of the first embodiment can be exercised.

Since the image data of the facial part of the user F and the image dataof the mother M and the child C can be clipped, for example, it ispossible to execute combination that image data of an arm part of theuser F is hidden behind the image data of the mother M and the child Cor combination that the image data of the user F is aligned behind themother M and the child C.

Although the above has described the combination that the background isremoved from the second image, a person separated from the backgroundmay be slimmed, subjected to a makeup, or formed as an avatar to becombined.

Additional advantages and modifications will readily occur to thoseskilled in the art. Therefore, the invention in its broader aspects isnot limited to the specific details and representative embodiments shownand described herein. Accordingly, various modifications may be madewithout departing from the spirit or scope of the general inventiveconcept as defined by the appended claims and their equivalents.

1. An imaging apparatus comprising: a first imaging unit which isprovided on a front face of an apparatus main body and images a firstmain subject to obtain first acquired image data; a second imaging unitwhich is provided on a back face of the apparatus main body and images asecond main subject to obtain second acquired image data; a clippingunit which clips image data of the second main subject from the secondacquired image data obtained by imaging of the second imaging unit; andan image processing control unit which combines the first acquired imagedata with the image data of the second main subject clipped by theclipping unit to meet combining conditions determined by the first imagedata.
 2. The apparatus according to claim 1, wherein the second imagingunit images the second main subject to obtain the second acquired imagedata, and the second acquired image data includes at least two pieces ofparallax image data having a parallax.
 3. The apparatus according toclaim 2, wherein the clipping unit clips the image data of the secondmain subject from the at least two pieces of parallax image dataobtained by the second imaging unit.
 4. The apparatus according to claim3, wherein the clipping unit clips image data of a portion having a highdegree of coincidence in the two pieces of parallax image data from oneof the two pieces of parallax image data.
 5. The apparatus according toclaim 1, wherein the image processing control unit comprises acombination determining unit which determines whether combination of theimage data of the second main subject clipped by the clipping unit ispossible with respect to the first acquired image data based on apicture composition of the first acquired image data including at leastthe first main subject as the combining conditions.
 6. The apparatusaccording to claim 5, wherein the combination determining unit comparesa position of the shaded portion of the first main subject with aposition of the shaded portion of the second main subject, judgeswhether the position of each shaded portion is inverted, and determinesthat the combination is impossible if the position is inverted.
 7. Theapparatus according to claim 5, wherein the combination determining unitjudges whether a combination space for the second main subject based ona position of the first main subject is present in the first acquiredimage data as the combining conditions, and determines that thecombination is impossible if the combination space is not present. 8.The apparatus according to claim 5, wherein the combination determiningunit judges whether the second acquired image data has been obtained byimaging the facial part if the second main subject is a person as thecombining conditions, and determines that the combination is impossibleif the facial part has not been obtained.
 9. The apparatus according toclaim 5, wherein the combination determining unit judges whether thefirst main subject and the second main subject have been imaged at thesame height position as the combining conditions, and determines thatthe combination is impossible if the subjects have not been imaged atthe same height position.
 10. The apparatus according to claim 1,wherein, if combining the image data of the second main subject clippedby the clipping unit with the first acquired image data is impossible,the image processing control unit informs that the combination isimpossible.
 11. The apparatus according to claim 1, further comprising:a face detection unit which detects the facial part of the first mainsubject from the first acquired image data, wherein the combinationdetermining unit judges whether combination of the image data of thesecond main subject clipped by the clipping unit is possible based onthe position of the first main subject detected by the face detectionunit.
 12. The apparatus according to claim 6, further comprising: aposture sensor which detects a posture of the apparatus main body,wherein the image processing control unit judges whether the first mainsubject and the second main subject have been imaged at the same heightposition based on the posture of the apparatus main body detected by theposture sensor.
 13. The apparatus according to claim 12, wherein theposture sensor comprises an acceleration sensor or an orientationsensor.
 14. The apparatus according to claim 1, wherein the imageprocessing control unit performs at least one or both expansion andcontraction of the clipped image data of the second main subject andcombines the image data of the second main subject with the firstacquired image data.
 15. The apparatus according to claim 1, wherein theimage processing control unit is configured to move the clipped imagedata of the second main subject in the first acquired image data. 16.The apparatus according to claim 1, wherein the first imaging unitimages the first main subject, and the second imaging unit images thesecond main subject including a photographer who holds the apparatusmain body to perform a photography operation.
 17. The apparatusaccording to claim 16, wherein the first imaging unit and the secondimaging unit perform imaging at the same time.
 18. An imaging systemcomprising: a first imaging unit which is provided on a front face of anapparatus main body and images a first main subject to obtain firstacquired image data; a second imaging unit which is provided on a backface of the apparatus main body and images a second main subject toobtain second acquired image data; a clipping unit which clips imagedata of the second main subject from the second acquired image dataobtained by imaging of the second imaging unit; an image processingcontrol unit which combines the first acquired image data with the imagedata of the second main subject to meet present combining conditions,thereby creating combined image data; and a combination determining unitwhich makes at least one of a judgment upon a picture composition of thefirst acquired image data including the first main subject as thecombining conditions, a judgment whether a combination space for thesecond main subject is present in the first acquired image; and ajudgment whether combining the image data of the second main subjectclipped by the clipping unit with the first acquired image data ispossible.
 19. The system according to claim 18, further comprising: aface detection unit which detects the facial part of the first mainsubject from the first acquired image data, wherein the combinationdetermining unit judges whether a space for combination of the imagedata of the second main subject clipped by the clipping unit is presentbased on the position of the first main subject detected by the facedetection unit, and judges whether combination of the image data of thesecond main subject is possible based on a result of the judgment. 20.An imaging method using an imaging apparatus, comprising: imaging afirst main subject by a first imaging unit provided on a front face ofan imaging apparatus main body to obtain first acquired image data;imaging a second main subject by a second imaging unit provided on aback face of the imaging apparatus main body to obtain second acquiredimage data; clipping image data of the second main subject from thesecond acquired image data; combining the first acquired image data withthe clipped image data of the second main subject to create combinedimage data; detecting the facial part of the first main subject from thefirst acquired image data; and judging whether combining the image dataof the second main subject clipped by the clipping unit with the firstacquired image data is possible based on a position of the facial partin the creation of the combined image data.